Services for Data Access and Data Processing on Grids

An increasing number of grid applications manage data at very large scale, of both size and distribution. In this paper we discuss data access and data processing services for such applications, in the context of a grid. The complexity of data management on a grid arises from the scale, dynamism, autonomy, and distribution of data sources. The main argument of this paper is that these complexities should be made transparent to grid applications, through a layer of virtualization services. We start by discussing the various dimensions of transparent data access and processing, and illustrate their benefits in the context of a specific application. We then present a layer of grid data virtualization services that provide such transparency and enable ease of data access and processing. These services support federated access to distributed data, dynamic discovery of data sources by content, dynamic migration of data for workload balancing, parallel data processing, and collaboration. We describe both our long-term vision for these services and a concrete proposal for what is achievable in the near term. We also discuss some support that grid data sources can provide to enable efficient virtualization. GFD-I.14 Vijayshankar Raman, IBM Almaden Research Center DAIS Working Group Inderpal Narang, IBM Almaden Research Center Chris Crone, IBM Silicon Valley Lab Laura Haas, IBM Silicon Valley Lab Susan Malaika, IBM Silicon Valley Lab Tina Mukai, IBM Silicon Valley Lab Dan Wolfson, IBM Silicon Valley Lab Chaitan Baru, San Diego Supercomputer Center